Tag
17 articles
NEEDLE is a dynamic search benchmark that rebuilds its query set every hour to prevent answer leakage and provide realistic performance evaluation of search systems.
Google DeepMind is pioneering a double-blind evaluation process for AI models using Confidential Space to ensure unbiased benchmarking. The initiative, tested with a Gemini Flash Lite model and the Singapore AI Safety Institute, could redefine how AI systems are assessed.
OpenAI announces enhanced cybersecurity measures following third-party evaluations that identified potential vulnerabilities in its AI models. The company outlines new safeguards to strengthen AI model testing and security protocols.
This article explains how AI models can be manipulated through adversarial prompts to bypass cybersecurity evaluations, highlighting critical safety concerns in frontier AI systems.
Learn to build a research agent framework similar to Perplexity's WANDR benchmark that evaluates AI systems' ability to discover evidence and provide verifiable sources.
Sixtyfour's innovative 'Eval Stack' approach challenges traditional AI development by rigorously evaluating agents against expert benchmarks, ensuring accuracy over assumption.
AI leaderboard platform Arena has reached a $100 million valuation after just a year, transitioning from a free tool to a commercial service in September.
Learn about the In the Weights score, a novel AI evaluation metric that analyzes neural network parameters to predict model performance and optimize training.
LifeSciBench is a comprehensive benchmark developed by OpenAI that evaluates AI models on real-life science research tasks, focusing on reasoning and decision-making rather than simple recall. It uses expert-authored rubrics to assess how well AI systems can handle complex scientific workflows.
OpenAI shares a comprehensive playbook to guide third-party evaluations of advanced AI systems, focusing on capabilities, safeguards, and validity.
This explainer explores Anthropic's BioMysteryBench, a new AI evaluation framework designed to test large language models in bioinformatics. It examines how the benchmark works, why it matters for AI development, and what it reveals about AI capabilities in specialized scientific domains.
Galtea raises $3.2M to help enterprises test AI agents, addressing the gap between demo and production performance.
